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Building Agency Capacity for Evidence-Based Policymaking 


State education agencies (SEAs) are increasingly under the gun to boost their 
effectiveness and advance student outcomes. This means chiefs and their deputies 
have to make the most of their resources and better understand how well existing 
programs and policies are (or are not) working in the field. 

To do just that, many state leaders are looking to grow their agency's capacity for 
generating, evaluating, and using evidence. In the fifth volume of The SEA of the Future, 
we explore how state education agencies can bolster their ability to use research 
and data to drive key spending, policy, and program decisions. We draw on the 
experiences of agency staff from Massachusetts, Michigan, and Tennessee, as well as 
the work of the Regional Comprehensive Centers. 

Carrie Conaway (Massachusetts Department of Education) describes why evidence- 
building is important, what types of research activities SEAs might focus on, and how 
to maximize research impact. As the head of the Office of Planning and Research in 
Massachusetts, Conaway offers SEA leaders concrete examples of how research can 
guide agency work in key areas like teacher evaluation and expanded learning time. 

Nathaniel Schwartz (Tennessee Department of Education) describes how to build 
a research team within the SEA and details the strategies his agency uses to make 
findings user friendly (and therefore, more likely to actually be used). Dr. Schwartz 
cites specific examples of his agency’s work to describe how evidence plays different 
roles in policymaking, from informing policy design to implementation and evaluation. 

Venessa Keesler (Michigan Department of Education) discusses how states can 
strategically leverage external research partnerships to supplement the SEAs’ own 
capacity to conduct research and advance evidence-based policy. Keesler draws on 
her experience working with external researchers in Michigan to describe how SEAs 
can make the most of partnerships. 

Finally, in an audio essay with three Regional Comprehensive Center leaders, we 
discuss how states can use RCC resources to better connect research and policy. 
Readers can see a transcript of that conversation at the end of this volume or listen 
to the discussion in a podcast. 

This volume includes practical tools SEAs can deploy around research and data. Ms. 
Conaway’s essay features a sample policy analyst job description and a research 
office organizational chart. Dr. Keesler’s essay includes a case study of a research 
partnership — the Michigan Consortium for Educational Research. Finally, our 
conversation with RCC leaders surfaced a blueprint for states looking to create a 
research office (based on Nebraska’s work with the North Central Comprehensive 
Center). 

Through these essays, the accompanying tools, and the more comprehensive 
supports the BSCP Center provides, we aim to give SEAs a solid foundation on which 
to build capacity for evidence-based policymaking. 
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In spring 2014, my agency, the Massachusetts Department of Elementary and 
Secondary Education, was at a turning point. 1 We were nearing the end of our 
first year rolling out our new educator evaluation framework for teachers and 
principals, with about half our districts having gone through an initial phase 
and the other half set to begin. While we had general statewide agreement 
that the old evaluation system didn’t do a good enough job differentiating 
strong, capable, and weak performance in teachers and principals — or 
identifying where educators could improve — the first year of implementation 
had been challenging. Districts struggled with training staff on the new 
model’s components, finding time for evaluators to conduct and document 
classroom observations, and identifying appropriate student growth measures. 
Anecdotal concerns surfaced that some teachers were more likely to get a 
positive evaluation simply due to the nature of the grade levels, subjects, or 
students they taught rather than their actual performance, thus creating a 
perception of unfairness. We planned to use a May 2014 statewide meeting 
on educator evaluation to acknowledge the new evaluation system's challenges 
and districts’ good-faith efforts to implement the new system, but also to rebut 
misperceptions where we had evidence that they were inaccurate. 

Fortunately, we had invested in research and evaluation projects to inform our 
implementation of the evaluation initiative: that meant we had solid evidence to 
share while we addressed districts’ challenges head on. We had examined the 
statewide distribution of performance ratings and compared them to student 
growth measures on our state assessment to understand how well the educator 
practice ratings aligned with the measures of educator impact on student 
growth. We had contracted for a large-scale program evaluation of the new 
system, including a representative sample survey of teachers and principals, 
as well as case studies and focus groups in a set of representative districts 
carrying out the new framework. With the survey conducted just a few weeks 
prior to the May meeting, we pushed for preliminary survey data so we could 
share new information on how educators perceived this major state initiative in 
time for the statewide event. 

As a result, my agency’s commissioner was able to share real-time data with 
the field in his keynote address. From our internal analyses, he could show 
that the ratings distribution was bell curve-shaped, with the majority of 
educators rated “proficient” on the summative performance rating, a measure 
of educator practice. This sharply contrasted with other states, where most 
educators rated in the highest performance level; our data demonstrated that 
evaluators were taking seriously the goal of better differentiating teaching 
performance. From the alignment analysis, he could show that higher 
professional practice ratings were associated with higher impact on student 
learning, supporting the rating system’s validity. The external evaluation, 
meanwhile, found that most educators felt they had received sufficient training 
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on the system as well as timely and helpful feedback from their evaluators. 
Importantly, it also showed that while many educators had concerns that the 
system overall was unfair, nearly 90 percent of those evaluated felt that their 
own evaluation had been fair. The commissioner used these research findings 
to acknowledge the work our districts had done to implement the system with 
fidelity, to show that the new evaluation system had a statewide impact, and to 
begin to alleviate educators’ anxiety about the system. Our research investment 
in this key state initiative positioned us to start to shift the state dialogue about 
the new evaluation system and to better support our statewide implementation. 

State education leaders face many choices about whether and how to invest in 
research and how to structure these inquiries to best support the state’s policy 
strategy. This essay will focus on where to begin: why research is valuable, what 
types of research activities state agencies might focus on, how to ensure the 
research is high impact, and how much it might cost. 


WHY RESEARCH IS VALUABLE 

Our story is just one example of how research has informed our state’s 
implementation of educator evaluation. We also relied heavily on research 
to develop our initiative in the first place. We combined the best evidence 
from existing research literature with stakeholder feedback to establish the 
program’s broad framework. Educators receive a professional practice rating 
of “exemplary,” “proficient,” “needs improvement,” or “unsatisfactory,” 
undergirded by a five-step cycle of self-reflection and goal setting. They also 
receive a separate rating that gauges their impact on student learning, which 
involves multiple measures of student achievement and growth over at least 
two years of data. Research also informed many decisions on the detailed 
regulatory requirements. The inclusion of staff and student feedback as a 
required data source for the professional practice rating was influenced by the 
Measures of Effective Teaching study, which showed these data provide insight 
on an important dimension of educators’ practice that other data sources 
fail to fully capture. 2 And our decision not to require a set percentage of an 
educator’s rating to come from their impact on student learning was based 
in part on researchers’ cautions that these estimates are often imprecise 
at the teacher level. 3 Findings from our commissioned program evaluation 
have shaped our technical assistance to districts: the state’s 2014-15 school 
year priorities on district-determined measures of student growth, ratings 
calibration across educators, and human resources practices all stemmed from 
evaluation findings indicating that these were areas where educators felt they 
needed more support. 4 We have also deployed our internal analytical staff to 
monitor implementation, identify problem areas early, and redirect resources 
and assistance as needed. 
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But our agency’s research investment goes well beyond educator evaluation. 
Other recent projects have examined implementation of our 2010 curriculum 
frameworks in English language arts and mathematics, our school and district 
turnaround programming and supports, placement patterns of low-income 
students into special-education programs, the impact of the state’s college 
merit scholarship program, and the impact of career and technical education 
on students’ academic outcomes. Some of our research has been done 
internally, some by university researchers, and some by professional research 
and evaluation organizations. In all cases, findings are shared with the field and 
incorporated into the lifeblood of the agency’s decision-making processes. 

Why have we spent so much time and effort on research? Because it makes our 
work better. It demonstrates to the field our commitment to gather and weigh 
their input and perspective as we implement major initiatives — and it does so 
in a systematic and democratic way so we do not hear only from those with the 
influence and power to communicate directly with state leaders. As our opening 
example demonstrates, research allows us to reframe public discourse about 
our initiatives and understand where the field faces challenges and needs more 
support. It helps us stay abreast of implementation challenges as they arise 
so we can adjust course as needed. It informs our strategic planning, both 
early on to help identify priorities and later to monitor rollout and outcomes. 
And it allows us to prioritize our financial and staff resources, determining 
what technical assistance is most valuable to the field and what resources can 
shift over time to programs with the greatest evidence of impact. Through 
all these mechanisms, research helps us continuously improve our program 
implementation to maximize results for our state’s educators and students. 


WHAT RESEARCH CAN DO 

Research encompasses a broad range of activities and inquiries that state 
education agencies can pursue. 5 But all of them help support continuous 
improvement of policy implementation using various lenses, methodological 
approaches, and levels of sophistication. 

Among the options are: 

• Literature reviews. Particularly valuable in the early phases of policy 
development or at key decision points in a program’s implementation 
strategy, literature reviews let us examine what is already known about 
a particular policy approach’s likely impact. This can help narrow policy 
options to those most likely to succeed and can flag potential blind spots 
or opportunities to expand or refine the program. The federal government 
offers several useful resources: the What Works Clearinghouse reviews 
recent research on effective educational practices, the Regional Education 
Laboratory program has a reference desk service to provide literature 
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reviews to education stakeholders, including state education agency staff, 
and the Comprehensive Center program provides technical assistance to 
states focused both regionally and on key education policy topics. 

• Descriptive analyses. Simple descriptive analyses — percentages, 
averages, medians, and the like — are the bread and butter of many policy 
analyses, with good reason. They are straightforward to calculate and 
widely understood by stakeholders. For years in Massachusetts, there 
was widespread knowledge of and concern over our public colleges’ high 
remediation rates. But in 2008, we established for the first time that 37 
percent of our state’s public high school graduates who enrolled in our 
public colleges were taking at least one remedial course in their first 
semester. 6 This specific, concrete link between students moving from 
the public K-12 system to the public college system garnered far greater 
attention to and concern about our students’ college and career readiness 
than the general statewide data previously available. It ultimately resulted 
in changes to the state’s four-year college admissions requirements and 

a greater focus on the rigor of the high school curriculum. And it gave 
us a yardstick against which to measure our progress toward reducing 
remediation needs. 

• Comparative analyses. Analyses that allow states, districts, and schools to 
compare themselves to one another and identify their strengths and areas 
for improvement are even more powerful. The Massachusetts research 
office produces a set of District Analysis and Review Tools (DARTs) that 
use enrollment and demographic data to identify the ten most similar 
districts for each district statewide, then display a range of simple 
charts and tables on student performance and other outcomes for these 
comparison districts. Identifying relevant comparison groups at the state 
level is particularly important, as it is hard for districts to know which 
others are most like themselves; the most similar district demographically 
might be on the other side of the state, far beyond the local district’s 
radar. Analyzing at the state level rather than the district level allows local 
educators to focus on their most important work: spotting opportunities to 
strengthen their practice. 

• Longitudinal analyses. The last decade’s expansion of state longitudinal 
data systems has facilitated much greater access to data on trends over 
time. The DART tools generally display the five most recent years of data 
on most indicators, and many Massachusetts data series go back ten 
years or more. The simplest version of a longitudinal analysis compares 
across cohorts: for example, last year’s 4th graders versus this year’s. More 
powerful analyses link the same individual students over time and follow 
their educational trajectories. Massachusetts has used these linked data to 
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create college enrollment “waterfalls,” starting with the 9th grade cohort 
and measuring what percentage of them reach key college milestones: 
graduating from high school on time, enrolling in college, and persisting 
in college (see Figure 1.) These are published in the DART Detail: Success 
After High School tool. 

Figure 1. Student Progression from High School Through 
Second Year of Post-Secondary Education 


First time ninth graders in 2007-08 and/or students in graduation cohort 201 1 



cohort 2011 5 years college enrolled in 

(Immediate Fall) college 


Your School 


Comperable 

School 


• Policy modeling. Research can also be beneficial in policy modeling: 
testing the potential impact of various policy options to help guide 
decisions. As Massachusetts developed our request for flexibility on certain 
provisions of the Elementary and Secondary Education Act, we modeled 
how many districts and schools would likely be affected by our proposals so 
we could reduce unintended consequences of our proposed accountability 
system changes. For example, we considered including student attendance 
in our accountability model, but once we tested it, we realized it made little 
difference in the determinations for most schools and decided to err on the 
side of simplicity. 

• Predictive modeling. Predictive modeling uses prior data on a student (or 
school or district) to predict their future outcomes. Massachusetts uses this 
approach extensively in its Early Warning Indicator System, which predicts 
for each Massachusetts student in grades 1 through 12 his or her likelihood 
of missing key academic milestones: proficiency on grade 3 reading, 
proficiency on grade 6 English language arts and mathematics, successful 
completion of all grade 9 courses, and on-time high school graduation. 
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Each student is assigned a “low,” “moderate,” or “high” risk of missing the 
milestone based on patterns of similar students in prior years. We produce 
both individual student-level and aggregate reports of these data and make 
them available to the educators serving each student through our secure 
online data system. These data allow district staff to better prioritize their 
resources and focus on students at greatest risk. The reports are among 
the most popular in our district data reporting tool. 

• Qualitative data collection. Quantitative analyses like those above are often 
fairly straightforward to run, particularly when they involve existing state 
data. But the nuance of the story — and the anecdotes that can prove so 
powerful when talking about policy to general audiences — typically comes 
from qualitative data collections such as interviews, focus groups, and 
observations. When done systematically, qualitative data collection can also 
shed light on implementation challenges. Our statewide educator evaluation 
implementation study included interviews and focus groups of educators 

in 12 case study districts. This qualitative data indicated that teachers of 
and staff in non-core academic subjects tended to look the least positively 
on the new evaluation system. When this finding became clear, our agency 
worked with four state associations representing these educator groups 
to develop additional resources to make sure the evaluation system was 
effectively applied to these roles. 

• Stakeholder perceptions. Collecting systematic, representative data on 
stakeholders’ perceptions on policy implementation — whether qualitative 
or quantitative — can reap enormous benefits for the state. It is a tangible 
way the state can demonstrate that it is listening and actively responding 
to criticisms and concerns. And it wards against paying too much attention 
to squeaky wheels when their perceptions are at odds with the majority 
opinion. Massachusetts collects these data through satisfaction surveys 
for superintendents and principals, statewide surveys of stakeholder 
perceptions on key initiatives, commissioned research, and other sources. 

• Causal analysis. We are often limited by available data to measuring 
only whether a particular policy is correlated with an outcome: for 
example, whether student achievement appears to increase along with 
implementation of a new policy. But some types of policies and programs 
lend themselves to what academic researchers call causal analyses: 
analytical approaches that can credibly claim that a particular intervention 
actually caused the observed outcome. 

Of these, the simplest analytically — but often hardest to implement — is a 
randomized controlled trial, in which participants are randomly assigned to 
either receive an intervention or to serve as a control. Because assignment is 
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random, program participants and non-participants are on average statistically 
the same before the intervention begins — so therefore any difference between 
treatment and control groups after the intervention must be due to the 
treatment (intervention or program) itself. But random assignment is often 
difficult in real-world policy settings, so researchers have invented other 
analytical approaches that approximate random assignment. These approaches 
include comparing outcomes for students who fell just above and below a 
qualification for eligibility in a program (for instance, a minimum GPA or an 
income threshold), comparing the trajectory of student outcomes before and 
after a policy was implemented, comparing outcomes for “treated” students 
versus other students with similar measurable characteristics, and so forth. 

When executed well, all these approaches allow us to be more confident that 
any difference we see between the treatment and control or comparison group 
is due to the intervention itself, not other factors that might have changed 
simultaneously. This can make for convincing evidence to share with legislators 
and funders. But these studies can also be complex and difficult to explain and 
can sometimes limit the generalizability of the findings. They are particularly 
worthwhile when a policy area is highly politicized and a definitive study is 
needed to address rebuttals, or when a program is new so it is easier to set up 
a random-assignment mechanism to study its impact. 

In Massachusetts, probably the most influential research of this kind has 
been the work of the School Effectiveness and Inequality Initiative based 
at the Massachusetts Institute for Technology, which has conducted a 
series of studies examining the impact of the state’s charter schools on 
student outcomes by using the school lottery process as a type of random 
assignment. 7 This work has convincingly demonstrated that students enrolled 
in charter schools in our state’s urban areas are achieving remarkable 
performance gains relative to similar students who applied to charter schools 
but did not gain admittance — and that those gains are greatest among the 
most disadvantaged students. 

• Cost analysis. It is valuable but often quite challenging to determine the 
actual cost of running a program, particularly when much of the cost 
comes from reallocating existing staff to the new work. Research can shed 
light on this through resource cost analyses that tie individual salaries 
and other expenditures back to the programs they support. My agency 
has used resource cost analysis to examine the cost of implementing 
Expanded Learning Time, a state grant program that provides $1,300 per 
pupil to about 20 schools per year to implement a longer school day. We 
learned that the per-pupil program cost varied from approximately $1,500 
to $4,300 depending on how districts chose to pay for the additional 
time required for staff and external partners to cover the longer day. 8 We 


10 


THE SEA OF THE FUTURE | VOLUME 5: Building Agency Capacity for Evidence-Based Policymaking 



Better Policy Through Research: Pursuing High-Impact Research in State Education Agencies 


have used this insight to develop guidance for new schools entering the 
program. In cases where it is possible to measure a program’s impact, 
we may be able to go a step further and analyze the program cost relative 
to its impact. Such cost-benefit analyses are infrequently done but are 
increasingly important as state and district leaders look to maximize 
their K-12 education investments. 9 Several research and technical 
assistance organizations are leading the way in this area by developing new 
methodologies and tools for districts and states to analyze how effectively 
they are using their resources. 

CONDUCTING HIGH-IMPACT RESEARCH 

With myriad options for analyses, it is crucial that states make smart choices 
about the research they choose to support. The most important consideration 
is selecting the right program for study. Principally, the program should be 
visible and sit squarely on the agency’s strategic agenda so that learning more 
about its implementation and impact will help drive agency priorities forward. 
Further, strong candidates for research are policies or programs that are 
malleable (potentially adjustable on the basis of findings) and durable (likely 
to be sustained for at least a few years), because this creates conditions where 
change is possible. If more information about a program’s implementation or 
impact will not affect decisions about its course — for example, if it has strong 
support from funders or stakeholders in its current form — or if the program 
is likely to be short-lived, then further study is likely not worthwhile. That said, 
if the state could conduct a causal analysis, that methodological rigor might 
counterbalance the voices supporting the program. But this is as much a 
political consideration as a technical one. 

Almost as important as selecting the right program for study is designing 
the right set of products from the research. Too often when state education 
agencies commission research, they require researchers to produce a 
lengthy document describing the program in great detail and including 
every imaginable analysis of its impact. This research often comes at the 
end of an implementation timeline — well past the time when the program 
is malleable or when its durability can be secured. Massachusetts made 
this mistake with its five-year evaluation of Expanded Learning Time. We 
hired an external evaluation firm Abt Associates, to conduct a study of both 
program implementation and impact. 10 The firm interviewed stakeholders, 
conducted student and staff surveys, and analyzed state data to measure 
program outcomes, taking a comprehensive look at the program statewide 
each year. But we requested just a single deliverable: an annual report covering 
all findings. Each year the researchers had to wait until well into the fall for 
our state assessment data from the prior year to become available so they 
could measure student outcomes from the program. They then needed time 
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to analyze and write up their findings. As a result, we typically didn’t have a 
report on the preceding school year until around February of the following 
year — far too late to influence program design decisions. Further, the 200- 
page document put off all but the most dedicated program staff from 
reading it, further diminishing its usefulness. And though the researchers 
spent considerable time thinking about how to fairly and accurately measure 
the extent and variability of program implementation, those ideas rarely 
crossed over to inform our agency’s program monitoring work, let alone into 
tools districts or schools could use to assess their own implementation. Abt 
Associates did just what we asked them to do. Unfortunately, the report we 
explicitly asked for wound up not being useful. 

Having learned our lesson the hard way, we now require shorter, differentiated 
products to more quickly get research findings into the agency’s discourse. 

We rarely commission the classic end-of-year tome unless the legislature or 
a grant funder requires it. We only “do the tome” when a program is ending 
or substantially changing course and we want definitive documentation 
of the program’s history to date. We now ask the researchers we hire to 
produce smaller, quick turnaround reports immediately after pieces of data 
collection are completed so that we can learn from their work in near real 
time. These might be internal memos, briefings for program leadership, or 
short summaries of findings intended for district audiences. Even when we 
are simply sharing data with academic researchers, rather than explicitly 
commission their work (see box), we require them to produce short, field- 
oriented summaries of their research papers that we then share with agency 
staff, superintendents, and principals. 
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Working With Academic Researchers 

University-based researchers are a great resource for conducting high- 
quality research. Academics interested in education policy want their 
research to be used and are often eager to contribute to state agency 
work. They can identify clever ways to answer research questions with 
existing data or new questions you had not thought to ask. They often 
can develop a more rigorous research design than your own staff. They 
can be particularly valuable in politically charged waters, for their 
work will often be viewed as more independent and even-handed. And 
if graduate students are involved in a project, investing time in them 
can pay off in a stream of work for years to come as they become 
familiar with the state’s data and policy context. 

Working with academics, however, also comes with some challenges. 
The academy values a different skill set than the policy sector. 
Academics are expected to narrow their focus to a very small question 
and expend as much time and resources as needed to answer it; 
policymakers must view each question in a broader context and likely 
face trade-offs in how much time and effort to expend across different 
program areas in a dynamic, relatively fast-moving environment. 
Similarly, complaints about academic writing are legendary; 
sometimes it seems the academy’s lingua franca is the Greek 
equation. But most challenging is the subset of researchers who 
view their interaction with a state agency as an opportunistic request 
for data with no real spirit of collaboration or partnership to answer 
questions of mutual interest. 

State agencies seeking to work with university-based researchers 
would be wise to clearly set expectations: What is the expected 
timeline for the work? Does the state want briefings on preliminary 
results along the way or an opportunity for agency staff to work 
alongside the university-based researchers? Should the researchers 
plan to present their findings at state board meetings or other public 
forums? An up-front investment in clarifying roles and responsibilities 
goes a long way toward establishing productive, enduring 
partnerships. 
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We also carefully examine variation in program implementation to gain 
insight into the conditions under which programs work best . 11 To this end, we 
generally require that researchers produce school- and district-level results, 
not just statewide aggregations, with the specific schools and districts named 
whenever confidentiality is not a concern. We also break down results for key 
subgroups, such as student demographic groups or types of educators. This 
gives us much better insight into the context of the findings and informs our 
technical assistance plans. 

Finally, we increasingly look for opportunities to turn researchers’ data 
collection methods into concrete tools for districts. For example, the research 
team analyzing our educator evaluation implementation came up with a 
methodology for assessing whether educators’ evaluations met various criteria 
of our statewide evaluation system; for example, establishing goals, providing 
effective feedback, and so forth. We are turning this into a toolkit for districts 
to do their own self-assessments, more efficiently leveraging the dollars 
invested in that research work. Similarly, we have shared with districts the 
questionnaires we used for our statewide analyses so they can gauge their own 
educators’ perceptions of the evaluation system. 


RESOURCES NEEDED 

To successfully build a central role for research in policy making, sufficient 
agency resources must be dedicated to the work. A key resource of any state 
education agency is its longitudinal data system. This can be used both by 
internal staff and external researchers to answer key policy questions. For 
internal staff, access to these data and training on how to properly analyze 
them may be needed to make the most of the data set. (See the “Getting 
Started” section regarding building an internal research team.) Since external 
researchers need timely access to these data to conduct most studies, 
setting up systems to facilitate appropriate researcher access is an important 
investment in promoting research. The Federal Education Rights and Privacy 
Act allows state agencies to provide personally identifiable student information 
to researchers under certain conditions. State agencies should develop 
memoranda of understanding with researchers that detail those conditions and 
any additional requirements they may want. For example, states might want to 
require researchers to provide them with an advance copy of findings before 
public release or require them to use industry-standard encryption to secure 
data files. 

Another resource, of course, is staff time. Internal staff will need to carve out 
time away from other priorities to make time for conducting actionable policy 
analysis. Making sufficient space for this work in your analysts’ portfolios is 
probably the single most important thing a state leader can do to increase 
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the use of data and research in policy making. This might be made possible 
by automating or simplifying some required data collection or reporting 
activities or by growing the number of staff with analysis skills by retraining 
existing staff or hiring new staff. Your analysts will also need time to assist 
external researchers to ensure that these external parties are using state data 
accurately. A simple first step is to create a researcher’s guide to your state 
data, to save staff from repeatedly answering the same basic questions. This 
guide should include an overview of what data are available to researchers and 
under what conditions, links to data codebooks with details of how variables 
are coded and formatted, and any key business rules researchers may need to 
know to use the data correctly. 

It is helpful to have a designated research and evaluation coordinator or 
at least to have a staff person with those duties as part of a broader work 
portfolio. This person can help define and execute the agency’s research 
agenda and serve as a liaison for researchers to help them use state data 
effectively. But, equally importantly, they can also work with senior leadership 
and program staff to ensure that the agency gets the most possible out of 
the research work, whether conducted internally or externally. Because our 
research coordinator in Massachusetts has a background in both research and 
program implementation, she can speak credibly to both sides and serves 
as a valued connector and translator between the two. When we hire external 
researchers, she ensures our procurement documents include the right 
information for bidders to be able to design appropriate research projects, 
getting us more for our money. She also manages most of our evaluation 
projects to ensure that vendors produce high-quality deliverables and answer 
our program staff’s research questions. 

A common rule of thumb in hiring an external research firm or academic 
researcher is for an evaluation to cost about 5 to 10 percent of the program 
budget (lower budgets may be feasible depending on circumstances.) In most 
cases, the data collection strategy is the key factor driving costs, with three 
main considerations: 

• The nature of the research questions. Research questions that can be 
answered with quantitative analysis of existing data are least expensive; 
questions that require extensive interviewing, focus groups or other 
qualitative data collections cost substantially more. Questions that require 
classroom observations to answer are often the most expensive, since 
researchers typically need to visit a large number of classrooms to ensure 
a representative snapshot. That said, classroom observations can also offer 
the most valuable data since they give evidence of whether your policy 
or program is affecting classroom instruction — the ultimate aim of most 
education interventions. 
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• The structure of the program. Programs carried out similarly across all 
program sites are least expensive to study, since fewer sites need to be 
contacted to get a representative look. If program sites have flexibility 
about what or how to implement, the sample selection will need to be more 
complex and, therefore, more costly. 

• The desired representativeness of the sample. If all you want is a rough 
statewide picture on a particular research question, researchers may only 
need to collect data from a few districts or schools. But if for political or 
practical reasons you want to ensure that all districts can participate in the 
study, or you want the researchers to stratify their sample to ensure that 
different types of districts are included (e.g,, rural vs. suburban vs. urban, 
high- and low-performing, regions within the state), costs may increase. 

Another cost factor besides the data collection strategy is embodied in the 
classic trade-off: “Cheap, quick, and good: pick two.” You can always make a 
project cheaper if you are willing to sacrifice quality or speed. But if you need 
an answer quickly, or if the results need to be unassailable, costs will likely be 
higher. 

To give some specific cost examples, in Massachusetts we dedicated about 5 
percent of the state’s Race to the Top grant to external program evaluation and 
hired two additional analysts to support our reporting, analysis, and evaluation 
work. This gave us sufficient funds to do in-depth implementation analyses of 
our major Race to the Top initiatives, as well as run specialized studies on other 
select initiatives. Project budgets ranged from $40,000 for a one-time, small- 
scale analysis of how districts were using results from a statewide educator 
survey to a multi-year $625,000 study on educator evaluation, which included 
substantial qualitative components and a statewide representative sample survey 
of principals and teachers. A from-scratch, moderate-length statewide survey of 
all superintendents and principals we developed cost about $40,000 including 
survey development, deployment, and results reporting for respondents overall 
and broken down by several district types. When we re-administered a similar 
survey the next year and requested similar reports, the cost was about half that. 
We have generally found that a case study done well costs $20,000 to $30,000 
per site to look at implementation and outcomes from a single program. Those 
costs can quickly add up if you want case studies representing the range of 
districts or schools in the program, in terms of either district characteristics 
(e.g., urban/rural, size, etc.) or quality of implementation. 
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Apart from cost, another key resource that supports research in state agencies 
is agency leadership. Leadership’s role is to consistently signal that research 
is high-priority work that you want staff to undertake, and that you expect 
staff to use evaluation results and data analyses as part of your continuous 
improvement strategy. With competing demands for staff time, research can 
easily be relegated to the back burner and ultimately never completed. And all 
too frequently, after the research is completed, the results are left to gather 
dust. Leadership can drive a culture of interest in both the research findings 
and in using findings to drive agency and system improvement. Clear direction 
from the top is critical to ensure a state agency research program's success. 


GETTING STARTED 

If you are ready to start boosting your state’s research capacity, below are 
steps to get started. 


1. Build the Team 

You probably already have staff analysts who prepare your state’s federal 
reporting and other special data analyses. These staff may be helpful in 
conducting research, but do not assume the required skills are identical. 
Ideally, your research staff should be more than just number crunchers 
(see box for sample job description and Figure 2 for an organizational 
chart). They should substantively understand both research methodology 
and education policy, as well as data analysis. That said, staff do not 
need doctoral degrees; often master’s students are well qualified for state 
analyst roles. Look for candidates who have taken beyond the minimum- 
required graduate coursework in statistics or research methodology and 
who have worked previously as a research assistant or analyst (not just 
class research projects.) Demonstrated skill in writing about technical 
topics for a general audience is key; you don’t want your staff to produce 
work that you yourself (let alone stakeholders) can’t understand. In 
Massachusetts we require a writing sample that demonstrates analyst 
candidates’ skill in writing about data. Second-round interviewees do an 
exercise in which they analyze state data and produce a short report similar 
to what they might do on the job. The exercise requires that they make 
judgments about which data to highlight and clearly explain the results. 
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Sample Policy Analyst Job Description 

Job duties: 

• Analyze qualitative and quantitative data on the implementation 
and impact of the state’s education initiatives 

• Develop comparative, longitudinal analyses of state, district, and 
school performance 

• Conduct analyses to model the potential impact of proposed 
policy changes 

• Prepare reviews of recent research literature on key topics 

• Work with program staff to identify program goals, 
implementation benchmarks, and outcome measures to help 
evaluate their programs 

• Write memoranda, presentations, briefing materials, and other 
reports summarizing findings for program managers and senior 
executives 

Qualifications: 

• Strong quantitative analysis skills, preferably including prior work 
experience in statistical analysis and reporting 

• Successful completion of substantial coursework in statistics or 
econometrics, with coursework in multivariate statistical models 
and/or research methodology (preferred) 

• Ability to use information gathered from research and key 
stakeholders to identify effective policy levers and promote 
improved outcomes 

• Strong ability to write effectively for general audiences 

• Strong project management skills 

• Excellent interpersonal skills and collaborative and consultative 
approach 
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Figure 2. Massachusetts Department of Elementary and Secondary 
Education, Office of Planning and Research 



You may opt to invest in improving existing staff's research skills in addition 
to (or in lieu of) hiring outside. Most analysts who completed graduate work 
more than five years ago will not have had much exposure to today’s more 
sophisticated analytical techniques. My agency bridged this gap by hiring an 
advanced graduate student from a local university to teach seminars for the 
agency’s analysts on those techniques. Our goal was for our analysts to at 
least understand when such techniques could be appropriately used, even if 
the analysts were not employing the techniques themselves. Other options for 
strengthening your existing staff’s research skills are having them participate in 
national programs such as Harvard University’s Strategic Data Project, sending 
them to research conferences by organizations such as the Association for 
Education Finance and Policy or the American Education Research Association, 
or providing them with training offered by local universities or national centers 
like the Interuniversity Consortium for Political and Social Research. Staff 
might also develop formal partnerships with any external researchers the 
agency is working with to build their skills as they are embedded in their job. 

All these opportunities will expose them to and connect them with the broader 
education policy research world. 


2. Build the Agenda 

If you are just beginning your research work, start small. Pick just one 
program or policy to study, ideally one that meets the criteria described 
above: visible, malleable, and durable. Your analyst staff may help you 
further narrow options by determining which programs have existing data 
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or program information (therefore making them easier to study). Consider 
launching a pilot project; this allows you to embed an evaluation design 
from the program’s inception, ensuring a knowledge base is built on the 
program implementation and impact throughout the pilot period. As your 
agency knowledge and capacity grow to manage the work, more projects 
become feasible. Ultimately, you should build an agency research agenda 
that tightly aligns with your agency’s strategic plan, with each goal in your 
plan carrying a set of attached research questions and planned deliverables 
to answer them. This will help strategically guide your agency to improve 
program implementation and heighten impact. 

3. Organize the Data 

Your state longitudinal data system is a great research resource, but it may 
take effort to organize it in a way that allows researchers to easily use it 
for analysis. The first step is to document the available data and business 
rules, as noted earlier, to save time answering basic questions. Creating 
special research files that integrate data across sources and removing 
some identifying student data (such as name and date of birth) may be 
helpful. That way, as researchers request data or as internal research 
requests arise, your staff will have everything at hand rather than having 
to make special files each time. Finally, you should establish a standard 
data-use agreement or memorandum of understanding that all external 
researchers must sign to gain access to your agency data. 

4. Mobilize Resources 

Since your internal staff will be major contributors to your agency’s 
research, they should play a key role from the get-go in producing analyses 
to support your agency’s research agenda. That said, some research is 
best done by external partners, whether local universities, for-hire research 
firms, or the federal Regional Education Laboratory program (which offers 
research and data analysis to state education agencies). Nate Schwartz’s 
essay in this volume provides more details on how to make the “build or 
buy” choice and how to manage work most effectively. 

Consider applying for grant funding to support your research. The 
Institute for Education Sciences, part of the U.S. Department of 
Education, recently created several grant programs that can help launch 
or sustain collaborations between state education agencies and research 
organizations. In fiscal year 2015, available grants to support emerging 
partnerships topped out at $400,000 over two years; grants to support 
major research and evaluation projects hit $5 million over five years. Other 
local and national funders are increasingly interested in supporting research 
partnerships as well. 
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5. Formalize Feedback Loops 

All this research effort is for naught if it does not improve your agency’s 
work. In Massachusetts, we use the U.S. Education Delivery Institute 
strategic planning and implementation systems, which provide us with 
routines and structures for sharing research with agency staff and 
leadership. Even without this degree of formality, simple steps such as 
asking analysts or external researchers to write short summaries of their 
findings and provide timely briefings to leadership and program staff 
can go a long way to infusing research into the agency’s work. The key 
is consistently making clear to staff that you expect to have data and 
information available as you make decisions and that you expect them 
to use those same data to improve their own work. Having program staff 
help build the research agenda and questions for their work establishes 
continuous improvement as an agency goal and creates a way for it to 
actually happen. 

CONCLUSION 

State education agencies have much to gain from investing in their research 
capacity. Developing a research agenda that dovetails with the agency’s 
strategic direction and making thoughtful choices about how to carry out that 
research agenda gives agencies a better chance of having the right information 
at hand when they need to shift course or make major decisions. State 
agencies can also more credibly demonstrate to the field that their input makes 
a difference in how the agency operates. These factors make research critical 
to advancing productivity in K-12 education as decision makers become better 
equipped to make informed decisions and to shift resources to maximize 
impact. Ultimately, research supports a cycle of continuous improvement that 
yields better programs and smarter investments. 
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State education agencies (SEAs) have significantly expanded their reach in the 
past few decades, driven partly by standards-based reforms that make states 
the primary actor in school accountability and partly by educator evaluation 
initiatives that grant states new authority to define and measure instructional 
quality. 

This broader state role means that the decisions of today’s SEAs are more 
consequential and demand more thoughtful analysis than ever before. Yet few 
SEAs are meaningfully poised to respond to these demands. To date, 21 states 
have no defined research or analysis office; those that do vary widely in their 
capacity to take on strategic analysis. 

This creates a difficult dynamic. The call for evidence is only growing, as is the 
recognition that data matters for strategic decision-making. But because state 
agencies often have little capacity for such work, research and evaluation tends 
to get outsourced or avoided altogether. 

In 2012, the Tennessee Department of Education established an office of 
research and policy to coordinate internal and external research across 
the agency. Before this office was created, analytic capacity was scattered 
throughout the organization; no team had clear ownership over the work. The 
office’s establishment heralded our state’s intent to have outcome-focused 
research help drive organizational policy. But it also forced us to rethink how to 
conduct research and analysis within a state education agency. 

Many state departments often view rigorous research as a nice-to-have 
supplement, but in the meantime, decisions must get made with or without 
definite evidence. To change this, agency officials must be able to expect 
in-time analyses that offer clear direction in decisions where the path is not 
already determined. Success cannot be defined by the research alone but also 
by the extent to which the research office changes the agency’s trajectory in 
measurable and meaningful ways. 

As we have moved down this path in Tennessee, we have faced choices about 
how to take on the work most effectively. This essay aims to make these 
decision points explicit. What defines the research agenda? What are the 
products? And what structures make it possible to achieve measurable results? 
The decisions we have made are not necessarily right for every agency, but the 
questions are ones that each SEA will likely face in prioritizing research-driven 
practice. 
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RESEARCH THAT COUNTS: CRAFTING AN AGENDA 
TO INFORM THE POLICY CYCLE 


Researchers within the SEA confront an extravagance of data riches. Where 
most researchers struggle to gain access to a data set that might allow 
meaningful analysis, those of us in state departments stand at the base of a 
waterfall with data rushing past us from every angle. 

Ever-expanding data and a broad array of state initiatives make for more 
research questions than even the most heavily staffed research office could 
hope to answer. This creates a continual dilemma about how to identify the 
highest priority topic areas and how to decide where we can get the greatest 
payoff for our research investment. 

Policymakers and observers commonly call for ever more program evaluation. 
Although we in Tennessee field internal requests for impact evaluations more 
frequently than any other demand, we do not think research can focus only on 
program evaluation if we want the research to drive policy. Such a 
single-minded focus severely limits research’s influence over practice, as 
explained further below. 

Figure 1 represents a typical policy cycle within an education department. 

The cycle begins with policy design and moves through implementation and 
evaluation. But for most policies, the starting point is fairly irrelevant since the 
cycle constantly iterates among all three components. 

Figure 1. How Research Fits in the Policy Cycle 
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When our research and policy team meets with department members about 
possible analyses, the discussion often jumps almost immediately to questions 
about program impact (represented by the “evaluation” box in Figure 1 above). 
This is understandable since these meetings are usually with officials who 
have launched a series of policy initiatives and want to know whether these 
initiatives have made a difference. But confining the research office’s work to 
the back end of a policy means research cannot inform policy and practice at 
other key junctures, such as policy design and implementation. 

Impact analyses are necessarily backward-looking and often must take place 
considerably after the fact to be able to draw on outcomes such as state 
test scores and/or teacher observations. By the time an evaluation can be 
completed, a policy has often already been in place for several years and state 
officials are already thinking about an entirely new set of initiatives. Moreover, 
without careful setup on the front end of the policy roll-out, it can be difficult or 
impossible to determine the extent to which a change in outcomes represents a 
true causal effect of the policy. 

In order to influence policy on the front end, research within a state 
department must focus equally — if not primarily — on the other two boxes in 
Figure 1: policy design and implementation (see Figures 2 and 3 for concrete 
examples of projects and outcomes aligned with each type of question). The 
sections below expand on this and offer concrete examples of some of our 
most successful projects at each stage of the policy cycle. 

Forward-Looking Policy Design Analysis 

Within our Tennessee office, we often refer to work on the design end as 
“landscape analysis.” By this, we mean analysis aimed at giving department 
personnel a clearer picture of a particular issue in order to drive initial policy 
efforts and pilot strategies. 

For example, after the Tennessee legislature set aside a relatively small grant 
to fund pilot strategies aimed at increasing student success on Advanced 
Placement (AP) exams, we did a landscape analysis of AP testing across the 
state, focused on understanding the factors that had kept low the numbers of 
students who actually earned college credit through AP exams. We identified 
trends in the type of obstacles students faced in different schools; in some 
schools, AP-ready students were rarely placed into AP courses in the first place, 
while other schools did better at placement but had few students who chose 
to pay for and sit the AP exams. By analyzing the landscape, we could offer 
a framework for classifying schools that prompted a series of differentiated, 
small-scale pilots aimed at helping remove obstacles we identified. We are now 
evaluating these pilots and choosing which to expand in coming years. 
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Similarly, a simple investigation into the typical math course progressions 
that students followed in different areas of the state yielded the surprising 
fact that enrollment in 8th grade algebra I (versus 9th grade algebra I) had 
fallen over several years from nearly a third of the cohort to less than one-sixth, 
severely limiting the number of students reaching advanced math by 12th 
grade. 

This study started as a design-focused analysis to inform curriculum decisions 
for our division of curriculum and instruction, but its results triggered an 
immediate shift in school and district accountability policies. And it prompted 
prolonged discussion with districts about how to ensure schools make course- 
placement decisions that push qualified students forward rather than hold 
them back. 

Such design analyses primarily aim to better define the problem our SEA seeks 
to resolve. Done well, these analyses can sufficiently narrow the scale of the 
issue and make it concrete so the agency can ultimately make a real difference 
in the field. If students are not completing the prerequisites for advanced 
coursework, a statewide program aimed at training new calculus teachers will 
have little payoff. If schools are not counseling enough qualified students into 
AP classes, covering the cost of AP tests is unlikely to yield major benefits. 

The best landscape analyses also create the conditions for further research 
on the effectiveness of department programs. A well-defined problem coupled 
with deep knowledge about the range of challenges and outcomes across the 
state makes it far easier to propose defined initiatives to solve the problem. 
These can then be rolled out and rigorously tested in comparison to other 
proposed alternatives. In our agency, our initial forays into data landscapes 
often wind up driving the creation of future programs. These data landscapes 
then serve as necessary precursors to most of the research that follows. 

Ongoing Implementation Analysis 

In the same way landscape analyses provide the foundation for policy design, 
implementation analyses serve as a crucial benchmark of progress on the path 
toward program effectiveness. Yet implementation research is the SEA’s most 
difficult kind of research. State administrative data tends to offer little in the 
way of meaningful implementation feedback. And few state departments have 
the personnel to undertake prolonged, qualitative research. 

While our research office is constrained in assessing implementation, we have 
found that we can offer valuable program development feedback through 
several relatively limited efforts that do not require the time and travel costs 
typically associated with deep, qualitative implementation studies. 
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Most importantly, we have benefitted from an effort launched several years ago 
with the Tennessee Consortium on Research, Evaluation and Development at 
Vanderbilt University to annually survey all teachers and administrators in the 
state. Reliable survey data allows us to monitor crucial changes in educator 
perceptions over time, as well as determine the extent to which the field is 
aware of and acting on state guidance around key initiatives. For example, the 
survey has tracked both the extent to which Tennessee’s teacher evaluation 
system is being faithfully implemented according to department guidelines, as 
well as how often teachers feel they receive feedback that helps them improve 
their practice. Data demonstrating success in the first realm and challenges in 
the second has helped push the department toward initiatives aimed at helping 
districts use evaluation as an improvement tool. Several efforts around teacher 
evaluation have been launched in ways that allow rigorous research on their 
effectiveness. 

We have also reaped payoff from “take-up” studies. These simple, descriptive 
analyses map the extent to which department initiatives reach their desired 
audience. For instance, when Tennessee offered leadership trainings through 
its regional offices, department officials noted considerable statewide interest, 
but few realized these trainings wound up reaching principals or assistant 
principals in 80 percent of Tennessee schools. Coupled with stats on the 
take-up of teacher trainings as part of the Common Core transition, as well 
as teacher subscription rates to regional reading instruction courses, this data 
allowed our office to map the extent to which state messages had penetrated 
the school level. 

Such studies (which we supplement, when possible, with the work of external 
researchers who have greater capacity for qualitative work) offer low-cost 
options to understand the ways state programs develop over time. Like the 
design-focused research described previously, these studies aim to bring 
research to bear on department policy long before program evaluation could 
provide results. 

Backward-Looking Impact Analysis 

Under the right circumstances, impact evaluations can be a smart use of 
department resources. But circumstances help dictate how helpful these 
evaluations can actually be. Below, we outline some considerations to keep in 
mind. 

Impact evaluations are much more likely to yield valid measures of a 
program’s effect if they are planned as part of a policy roll-out rather than 
after the fact. Pilot programs can often provide ready-made opportunities 
for meaningful evaluation, assuming that someone is thinking about these 
opportunities during the pilot set up. For instance, in Tennessee the law 
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required us to create dual-credit courses, offering high school students the 
chance to receive college credit by passing statewide end-of-course exams. 
Early on, it made sense to limit some of these courses to a select set of 
schools. We gathered a pool of interested schools and randomly selected some 
from each region to create our initial pilot sample. This kept the initial pilot a 
manageable size and created a gold-standard research design that would allow 
us to directly measure the effects of the program on student outcomes (as we 
would have some schools with the program and some schools without, as a 
control). Recognizing the research design’s strength, the Institute of Education 
Sciences (IES) awarded $2 million in 2014 to the project partners, including 
the Tennessee Department of Education, to support ongoing research. 

When stakes are high, we recommend passing impact studies to trusted, 
independent researchers to ensure results are buffered from organizational 
politics. Evaluations are judgments of program effectiveness, and therefore 
have winners and losers. Even the most research-driven organization will 
struggle at times to come to terms with negative evaluations of popular 
programs. In the case of the dual-credit courses study described above, we 
enlisted the help of nationally known researchers at the University of Michigan 
and the University of North Carolina, whose work in this area ensures the 
final findings will receive serious consideration from both practitioners and 
academic researchers. 

Although state departments administer many programs, usually only a 
handful are positioned for evaluation results to have an immediate impact on 
departmental strategy. In these cases, we advocate doing whatever it takes 
to allow rigorous research to weigh in on the strategy. For example, as part 
of its Race to the Top grant, Tennessee offered statewide summer teacher 
trainings through an innovative peer coaching system. As the grant wound 
down, department staff questioned whether the trainings — though popular — 
were worth future investment. Our team's rigorous impact evaluation found 
that teachers who attended the state trainings both received higher classroom 
observation scores and saw better student results on state tests. While the 
effects were not huge, they were enough to deliver a clear cost-benefit payoff, 
and they helped drive the decision to continue training teachers beyond the 
federal grant. 

Choosing the right program evaluations to undertake means choosing only 
those questions for which the department has: 

A. the right set of decision points; 

B. the opportunity to apply appropriately rigorous methods, and; 

C. access to data on meaningfully aligned outcomes. 
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Meeting this criteria demands constant communication between research staff 
and department leadership, which in turn requires explicit attention to the 
relationship between the department’s research and operational divisions. 


RESEARCH THAT MAKES A MARK: 
DRAWING ACTION FROM RESULTS 


Just as the agency’s research agenda needs to speak to the agency’s needs, 
so do research results. Asking the right questions and conducting the right 
analysis is only a piece of the puzzle. Too often, researchers see the analysis as 
the endpoint rather than the beginning. 

What does it take to make findings meaningful for department leadership? 

First, analyses must be done within the right time frame. This does not 
preclude a lag between project conception and results. Certain questions are 
likely to be meaningful for long-term department strategy; investing in long- 
term research on these topics can be worthwhile. Others will only catch the 
department’s attention if they are linked to particular decision points. The key 
is to fit the research to the time-frame rather than the other way around. We 
have seen projects our researchers thought fairly uninteresting take on outsize 
influence simply because they appeared at a time when people were hungry 
for our answers. Similarly, we have done other projects that our office saw as 
hugely important and watched the work languish because we developed the 
findings at a time when no one was ready to take ownership over the results. 

Second, research must be framed to speak to department policymakers. In 
other words, department personnel must be convinced that the findings relate 
directly to what they do. 

One cannot assume that the questions will speak for themselves. From 
the outset, the research must be framed to explicitly suggest the kinds of 
“takeaways” that various department groups might draw from it. In other 
words, researchers bear the responsibility to explain why findings in one 
direction or another matter for department policy. 

As findings develop, team leads within the department must be given time to 
process the findings and integrate them into their worldviews . 1 This can only 
happen through discussion and engagement. Again, it becomes incumbent 
on the research team to ensure that these department discussions are taking 
place — to ensure that they not just pass on findings but also launch a reflection 
on the results that will culminate in department action. 

Third, presentation matters. Often, this gets reduced to a discussion about a 
report’s length and the attractiveness of its graphics. In my job, I am frequently 
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reminded that department personnel have so much on their plates that they 
won’t bother to read anything that is more than a few pages long. Though often 
true, the simplification around report length sidesteps a lot of other important 
considerations that help make research influential within a state education 
department. 

After much trial and error, our office of research and policy has adopted several 
informal principles that guide the translation of initial research and analysis 
into a product that can drive policy. These are by no means revolutionary; they 
represent much of what people have said for years about actionable research. 
However, consciously abiding by the principles has pushed us to make our work 
both more digestible and more immediately impactful. 

In our view, presenting research to drive policy means constructing an 
actionable narrative, illustrated with meaningful figures, in order to build 
lasting frameworks that organize departmental thinking and policymaking 
within a particular area. In the space below, I describe these principles in more 
detail. 

A Clear Storyline: Policies often get driven by small-scale, anecdotal evidence. 
This can be frustrating, particularly for researchers, but it is not surprising 
when we consider that policymakers are necessarily in the business of 
communication and communication thrives on stories. 

We have come to believe that making the findings deliver the same 
communication power as individual anecdotes is key. Research results must 
function as stories that lead to a clear conclusion. If it is impossible to put 
together a list of bulleted takeaways that elucidate a series of potential project 
implications, the researcher has failed to take the final crucial step to make the 
research relevant. 

During our weekly research team meetings in Tennessee, we listen to research 
analysts present the studies they are working on and press them with pointed 
questions to succinctly convey what they are trying to say and what makes 
it meaningful. Through this process, we work to distill each project to the 
storyline that offers a clear reflection on department policy. Each eventual 
product comes with a clear set of driving ideas and implications. The point is 
not to force every product to only a single page of results, but rather to force 
every product to make clear takeaway points that convince both researchers 
and practitioners that the project has offered something concrete to facilitate 
department decision-making. 

Meaningful Figures: This is not a call for sophisticated infographics. 

Every moment spent deciphering a graph’s intricacies is a moment lost 
to understanding the graph’s implications for the department’s work. If 
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department staff are accustomed to stacked bar graphs created in Excel, 
there is no need for anything more. Eventually, as the audience expands, more 
complex graphic design is worth considering. But we have found that the initial 
pass should come in a language familiar to the audience. 

An exception to the rule of sticking with what people initially know is when 
certain types of difficult-to-understand graphs or figures can add value to 
multiple projects. In these cases, it can be worth investing the time to help 
decipher them for the given audience and regularly include these types of 
graphs across presentations, allowing department personnel to become 
accustomed to their look and feel. One example from our work in Tennessee 
comes from the common observation — that is not specific to Tennessee — that 
statewide averages tend to mask considerable variation at the district and 
school levels. To make this picture visible, we present bar graphs, such as 
Figure 2, where each bar represents a single district (or school). The example 
figure shows variation in district-level teacher retention. This kind of figure 
allows easy comparison between individual districts and the state average, 
it raises a series of questions about the districts (or schools) that fall at the 
extreme ends of the x-axis — and it can easily be altered to include more 
information, as in Figure 3, which compares different districts’ ability to retain 
their most versus least effective teachers. Simply because of the number 
of bars, such graphs can be initially confusing to a new audience, however, 
we have used them frequently enough across presentations to make them 
immediately comprehensible to our audience within the Tennessee Department 
of Education. 

Figure 2. Overall Retention Rates by District (All Teachers) 


Overall Retention Rates by District (All Teachers) 



■ Retained in Same School □ Retained in Same District ■ Retained in Other District 
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Figure 3. District Retention Rate Differences by Effectiveness 


District Retention Rate Differences by Effectiveness 


~ 25 



Frameworks for Anchoring Main Ideas: This principle has prompted 
considerable discussion across our research team, since the meaning of 
“framework” shifts from project to project. Yet increasingly we agree that the 
models and frameworks research can provide for approaching a policy dilemma 
often add as much value as the quantitative results that fit inside the research. 
In the best cases, the logic model offers organizing principles for defining an 
issue that can live on in people’s minds long after they have forgotten the exact 
numbers that accompanied the principles. 

For example, the earlier noted landscape study of Advanced Placement testing 
across our state developed the idea of an AP-ready student (identified by 8th 
grade test-score results) and classified schools into buckets based on the types 
of AP obstacles their students encountered. “Low preparation schools” were 
high schools whose feeder schools never produced enough AP-ready students 
to justify AP course offerings. “Low access schools” had AP-ready students but 
no AP course offerings. “Differential enrollment schools” had low-income, AP- 
ready students enrolled in AP courses at far lower rates than non-low-income, 
AP-ready students. 

The model included six buckets of schools in all, each of which defined an 
access problem in a particular way. What developed was both a concrete 
analysis — X percent of schools are low access schools, Y percent of schools are 
differential testing schools, etc. — and a broader way of thinking about targeted 
school supports. As we wrote in a sidebar to the AP report, “Too often, our 
policy solutions and interventions are crafted as one-size-fits-all policies that 
fail to differentiate based on the highly variable data coming from individual 
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schools and districts.” By creating a data-based diagnosis tool for school 
problems in the AP realm, the research work prompted a discussion on how we 
could use similar techniques in other circumstances to more accurately identify 
intervention needs. 

Our AP intervention organizing model, and others like it, have taken on a life of 
their own after initial presentation, recurring in a host of different discussions 
and prompting different department groups to consider their own work in 
new ways — even outside of the initial project arena. The right models make 
individual projects generalizable, providing new vocabulary and new guidelines 
for confronting recurring policy problems. 


STRUCTURING THE WORK 


The agency has to commit to make research possible. And this requires 
supporting arrangements: personnel, resources, and management structures. 
What does this look like and what does it take to make such support available 
within an SEA bureaucracy? 

Team Dynamics 

It is entirely possible for a state department to prioritize research without 
consolidating its researchers into a single team, but the group structure 
impacts the type of work that can happen. 

Before Tennessee moved to a single-office structure, department divisions 
carried out high-quality individual pieces of analysis, but no one was 
responsible for focusing on developing research to inform department-wide 
strategy. The old structure meant Tennessee lost the ability to deploy research 
as a surveying tool across programs and divisions. This in turn contributed to 
our department’s already silo-like nature. 

The lack of a unified research office also translated into little to no oversight 
of the external research partnerships formed during this time. Research 
partnerships tended to look more like one-sided transfers where Tennessee 
provided data and researchers carried out their own analyses for their own 
purposes. Giving a single team oversight over the research agenda has made it 
far easier to seek out partnerships that answer departmental needs rather than 
the other way around. 

At the same time, a separate research team brings its own challenges, the 
biggest being a lack of predefined connection with the research’s end-users. In 
Tennessee, we see building these connections as a discrete component of our 
work. Each research team member serves as a liaison to a department division, 
regularly meeting with division staff and seeking feedback on the direction of 
their research. 
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Equally important, research team members look for opportunities to perform 
“technical assistance” for their given division. Technical assistance here 
refers to the many data tasks that always need doing in a state agency — tasks 
that would not necessarily classify as research but that require some skill in 
manipulating data. By pitching in on data-related tasks for different divisions, 
research team members build the relationships that allow them to meaningfully 
connect their research to the department’s operational work. Without these 
personal relationships built on mutual benefit, our research team’s influence 
would likely be much diminished. 

One issue to note: It is easy for technical assistance requests to overwhelm 
the actual research and analysis work. In Tennessee we have found that the 
department has an insatiable need for people who can answer questions 
about data and combine files into new spreadsheets. The quantity of federal 
reporting always outstrips the quantity of individuals trained to create the 
necessary spreadsheets; demand for data to be sliced in new ways is constant. 
While it is important for the research group to provide immediate technical 
assistance benefit to different department divisions, it is equally important for 
the research office to be insulated from the department’s many operational 
data tasks. Without some buffering, the research group will soon struggle to 
keep up with the day-to-day work of compliance and reporting and fail to serve 
the strategic purpose for which it was created. 

Building and Financing the Team 

Making research relevant is difficult work. In Tennessee we have benefitted 
greatly from ready access to well-trained researchers from institutions like 
Vanderbilt University. Yet the people we look for are not only those with strong 
methodological skills (or even with the ability to deal with large and messy 
data sets). Instead we look for people who are both ready to engage in rigorous 
analysis and who value the “everything else” described in the above sections 
that must accompany research to make it relevant. 

Happily, the number of applicants who fit this description seems to be on the 
rise. Organizations like the Strategic Data Project have seen a steady increase 
in applications and their programs’ cohort size. The Institute of Education 
Sciences encourages training through pre-doctoral fellowship programs 
that require focusing on applied research and practitioner collaboration. In 
Tennessee, we have received a steady stream of well-qualified applicants for 
every open position and have benefitted from a series of interns from area 
Master’s in Public Policy programs who seek useful practicum experiences. 

But even with a strong research team ready to do the work and an agency that 
prioritizes research, we in Tennessee still face the reality of limited funding 
and staffing. One way we have been able to combat this is to explicitly place 
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our research in service of various federal cost objectives and thus parcel out 
federal dollars across individual researchers. Each member of our team logs 
hourly personnel activity reports noting the projects they have worked on and 
the program link. For example, at the end of the year, one team member may 
wind up having worked 20 percent of his or her time on Title II initiatives, 30 
percent on ESEA Title I initiatives, 40 percent on IDEA initiatives, and another 
10 percent on state activities. 

Even where federal funding might not be a possibility, the strategy of splitting 
funding for a research team across the areas where the analysts actually 
add value remains an important principle for us in Tennessee. Among other 
advantages, it forces the research team to actually make good on its value 
proposition. Since research team members are getting paid from buckets of 
money that other department teams control, our researchers must constantly 
ensure agency members feel like they receive direct benefit for the dollars 
spent. 

Balancing the Work Across Internal Personnel and External 
Partners 

Even among SEAs with strong research offices, the approach to external 
partnerships is often quite different. Some SEAs choose to rely on their own 
analysts to complete most of the work; others farm all projects out to research 
firms and academic partners. The place where an SEA falls on this spectrum 
has considerable implications for the type of staff needed to staff the research 
team. 

In Tennessee, we aim to place ourselves somewhere near the center point, 
claiming most of the quick turnaround and forward-looking analysis for 
our internal team, and then passing off longer-term evaluations to external 
researchers. Under this theory, external researchers serve a role that is 
meaningfully different than that of even the best internal staff members. 
Research partners offer an independent look at state initiatives, they bring 
new ideas into the department, and they often use more rigorous (and slower) 
methodologies than a rapid-response state team. The challenge that states face 
is finding ways to balance these gains with the additional challenges of taking 
research outside the agency. 

While it is simple enough to define a set of products for research that 
is created by an internal team of researchers, it is far more difficult to 
communicate these expectations to a varied set of research partners that are 
drawn from a wide set of research firms and academic institutions. 

Even the best of research partners do not operate within the agency, and so 
they will tend to be less adept at framing questions and findings in a way that 
feels relevant and influential than an internal team that communicates with its 
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constituents on a day-to-day basis. Moreover, research partners face different 
cost structures and incentives, all of which can interfere with demands of 
timeliness and/or presentation. 

Managing this process while containing the amount of time spent on this 
management has proved to be a considerable challenge for our team in 
Tennessee, and we feel like it is an area where we have much to learn from 
other states. Notwithstanding some striking successes with individual 
researchers, we continue to struggle to reach a point where the process 
of partnership — and the considerable cost of time and effort — feels like it 
consistently pays off in terms of research and policy impact. 


DEVELOPING THE WORK 


While we have logged several successes tying department practice to research 
in recent years, our team believes that our current structures fail to meet our 
agency’s needs in two central areas. First, we are not set up to track recurring 
metrics over time and to support repeated analysis of these metrics, even as 
we increasingly produce data points that we would like to track longitudinally. 
Second, we struggle to connect the dots across research projects (internal 
and external) within the same areas, contributing to a sense that we are 
undertaking a series of disconnected studies rather than producing definitive 
work within a few high-priority focus areas. This final section details our 
diagnosis of these missing pieces and our strategy for moving from where we 
are to where we hope to be. 

Our team often jokes that we only start new projects, never finish old ones. 
Successful work produces new metrics and new ways of looking at data that 
in turn generate further appetite for tracking these metrics over time. For 
example, analyses investigating the landscape of human capital management 
across the state (teacher retention, equity in student-teacher assignment, etc.) 
produced a series of data breakdowns within these areas that department 
personnel hope to keep tabs on over time, both at the state level to monitor 
progress on strategic priority areas, and at the district and school levels to 
target interventions and support district strategic planning. Yet if our group 
committed to producing yearly analyses or even yearly data reports following 
human capital metrics at the state, district, and school level — and if we 
were to meet similar demands in other areas such as discipline disparities, 
chronic absenteeism, and several others — we would quickly lose the ability 
to undertake new projects. Moreover, at some point, the work of tracking 
chosen metrics and analyzing these data starts to look less like research and 
analysis and more like progress monitoring — a different, though equally vital, 
organizational need. 
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To answer this need, we are investigating the possibility of creating a small 
offshoot group to help coordinate department-wide strategic progress 
monitoring. With assistance from each division, this team would help ensure 
that we follow the right metrics and use these as levers to drive our work. The 
goal would be to take the indicators and tools developed on the research side 
and put them into the hands of an expanding state audience. Over time, some 
metrics might develop into established ways of looking at state educational 
data, at which point they would migrate to our state’s online report card. 

Others may serve a particular need at a moment in time and then eventually 
drop out of use. Importantly, the team’s charge would not be to create 
platforms and structures just to monitor recurring strategic metrics over 
time (we expect that such data points on their own would be only marginally 
useful). Instead, the team would work closely within the department and with 
our regional offices to directly engage others in understanding these metrics 
and to draw on their expertise in order to surface new ideas for current state 
strategies. 

Our second major challenge has been to connect the myriad research 
efforts going on simultaneously through our office and external research 
partnerships. We are in the midst of two internal projects that look at the 
connection between our state teacher evaluation system, teacher feedback, and 
instructional improvement. Two other external research projects take on the 
same topic from different angles. 

Each of these studies is likely to produce interesting results and we hope 
each will drive agency action. But each project feels like an individual venture 
independent of the others, which we believe significantly lessens the total 
impact. We have not yet managed to create a system or process where we 
regularly look across current projects to ensure that each ongoing study takes 
advantage of the other researchers’ information. And we have not developed 
a way to tell the broader story — to our department or the public — of what we 
have learned in Tennessee from all the research on teacher evaluation and 
instructional improvement. 


CONCLUSION 


As our office continues to develop, we are looking for ways to make each 
research project contribute to a broader picture of educational practice and 
policy in Tennessee, potentially with help from outside research organizations. 
We aim to synthesize findings across our research portfolio to offer integrated 
recommendations about state and district action. But this work demands both 
a different skill set and a greater number of positions than our office can cur- 
rently fund. We join other SEAs in struggling to balance priorities as we identify 
new project goals. 
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ENDNOTES 

1. For a thoughtful discussion of the time and effort it takes to integrate 

research results into agency operations, see Melissa Roderick, John Easton, 
and Penny Bender Sebring, The Consortium on Chicago School Research: A 

New Model for the Role of Research in Supporting Urban School Reform 

(Chicago: University of Chicago, 2009). 
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INTRODUCTION 


State education agencies (SEAs) have increasingly shifted the focus of their 
work from compliance monitoring to performance management. Inherent 
in this shift is a need to use data and information on “what works” to drive 
decision-making. Leading states are not only investing in longitudinal student 
data systems that can help track key outcomes over time but also in research 
partnerships that allow states to make use of their data in sophisticated ways 
to advance state policy. This chapter focuses on these research partnerships — 
why they are needed, what it takes to build and sustain them, and the common 
challenges involved. 


As Nate Schwartz discusses in this volume, states can and should build their 
internal research capacity. But external research partners offer states unique 
benefits. By leveraging both internal and external resources, SEAs can more 
effectively and efficiently meet the demand for quality research. 

External partners give states unbiased and politically neutral research results 
that are independent of the state’s policy environment. They can add to 
(and complement existing) analytic capacity. They integrate both policy and 
academic approaches to analysis and problem solving. And they can provide 
needed specialized expertise to support state policy. Think of the SEA-external 
research partner relationship as a Venn diagram (Figure 1), with the overlap 
representing shared topics of interest. 

Figure 1. External Researchers Can Complement the SEAs' Research Needs 
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Flexible Source of Expertise 

Research is a specialty field: most researchers focus on just a few areas and 
become expert in them. Partnering with external researchers allows SEAs 
flexibility to get the “best of breed” in diverse expertise areas and skill sets. 
Depending on the question or task at hand, an SEA may need someone who is 
expert at randomized control trials, survey development, or quasi-experimental 
designs with longitudinal data. Alternatively, an SEA may want someone 
steeped in turnaround research, teacher induction practices, or school 
safety. Finding this breadth and depth from the limited number of in-house 
SEA researchers is impossible. When SEAs partner strategically with external 
researchers, they can match agency needs with the person (or organization) with 
the best mix of technical skills, content knowledge, and interest in policy work. 

In our Michigan agency, we faced research questions about the impact of 
school choice policies on sending and receiving districts that our agency was 
not methodologically equipped or politically positioned to answer internally. 

We partnered with a local university researcher who brought the right skills, 
interest, and profile as an independent observer with no vested interest in what 
findings the research revealed. In other projects, we have been able to partner 
with researchers armed with expertise in longitudinal data analysis, methods 
of estimating “effects” over time while controlling for many factors, and 
specific econometric modeling skills. 

Partnerships with external researchers also establish mutually beneficial 
connections between research, policy, and practice. An external research 
partner can serve an SEA as both a source of expertise and as a good critical 
friend. This can help both the SEA and the researcher cultivate a more 
nuanced, grounded, politically neutral, and long-range view of how to tackle 
a problem than might not be possible if each party worked alone. A prime 
example is the educator evaluation work going on in many states. States are 
tasked with building the educator evaluation system; researchers in many 
external institutions have been considering for decades the components of 
educator quality and how to measure those components. The conversation is 
enhanced when we have it together — SEAs need to know how to do this work 
and researchers need their theories to have real-world application. 

In my Michigan agency, an initial partnership with an external researcher 
interested in the impact of mandatory college-entrance exams evolved into a 
much more elaborate intervention strategy to improve the college matriculation 
of at-risk students. As the researcher was working through his initial questions, 
we were developing a postsecondary transition plan. This gave us a chance to 
try some new strategies and study their effectiveness at the same time. We 
were also able to connect the researcher with the Michigan College Access 
Network, creating a three-way collaboration that benefited all involved. We 
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now know not only the impact of mandatory college entrance exams, but we 
were also able to develop and offer to districts a suite of research-based, 
postsecondary supports designed to improve student outcomes. 


Benefits and Challenges of Research Partnerships 

Benefits: 

• Achieving flexible capacity based on need for methodological or 
topical expertise 

• Tapping in-demand talent that would otherwise be out of reach 

• Providing fruitful connections between policy, research, and practice 

Challenges: 

• Finding research partners with the right expertise, interest, 
availability, and skill sets 

• Finding resources to fund the research 

• Ensuring results are visible to internal and external stakeholders 


GETTING BEYOND THE CHALLENGES 
OF PARTNERSHIPS 


SEAs seeking to build productive research partnerships must overcome several 
challenges: 1) finding research partners with the right expertise, interest, 
availability, and skillsets; 2) finding resources to fund the research project, and 
3) ensuring results are visible to internal and external stakeholders. 

Finding and Developing a Research Partner Relationship 

An SEA’s first challenge is identifying a partner with an active research agenda 
in the agency’s area of interest. Sometimes this is as simple as looking for a 
researcher with a specific expertise; for example, if the SEA wants to identify 
best practices in literacy instruction, it can tap leading literacy experts working 
in local universities or other research organizations. 
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But often the SEA is interested in broader questions than specific specialty 
areas cover. For example, a question like, “What are the characteristics of high- 
performing schools?” could be addressed by research on whole-school reform, 
reading and literacy, culture and climate, or myriad other topics. But the SEA 
needs a partner capable of synthesizing all the relevant research literature, not 
just individual pieces. When the SEA requires broader expertise, the agency can 
form “umbrella” partnerships with an institution so the state can tap both a 
range of expertise, from broad to narrow. 

Another challenge is balancing the reward structures of external researchers, 
which differ from those of SEA staff. SEAs focus on (and are judged on) 
addressing policy problems and ensuring implementation fidelity. Researchers 
typically focus on (and are judged on) publishing articles in peer-reviewed 
journals, where journalistic standards and specialization can limit the research 
findings’ applicability to real-world policy problems. 

In Michigan, we have worked to build a bench of external researchers with the 
right interests and skill sets. While an SEA can mine existing connections to 
build such a bench, continually building new connections with the research 
field through conferences, like the Association for Education Finance and 
Policy, is key. In the Michigan Department of Education, our research staff 
makes time to attend research conferences twice a year. We prepare for 
these conferences ahead of time in order to maximize their utility, identifying 
researchers to connect with and relevant panels to attend. 

We also cultivate ties with graduate students, who will eventually move on to 
full-time research roles. Together with the University of Michigan, we sponsor 
the Education Data Fellows program, connecting graduate students interested 
in working with SEAs to expand their technical and policy skills. This program 
not only boosts our internal research capacity, it also deepens our connections 
to future researchers. 

In Michigan, we have faced a partnership constraint around aligning timelines. 
Unless an SEA has ongoing research partnerships, or a stable of “on call” 
researchers, it can be challenging to get a research partner on board, get them 
up to speed, provide the data, and get results in a policy-relevant timeframe — 
which is definitely shorter than a typical research-relevant time frame. External 
partners need to be willing to produce on firm deadlines and produce exactly 
what the SEA needs. 

To better manage these challenges, in Michigan we now try to start all 
partnerships with a scoping meeting that includes all impacted program and 
research staff. We use this meeting to establish key milestones and products. 
We give each of our priority partners firm internal deadlines and a single 
agency point of contact, rather than trying to manage the relationship in a 
more informal and ad hoc way. 
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Research partners who want to work with state administrative data need to 
have strong quantitative skills and reasonably good detective abilities. The 
SEA administrative data sets are a departure from what many researchers are 
used to working with, such as those from the National Center for Education 
Statistics or small survey data sets. Our SEA has millions of records amassed 
over decades, and the way things were collected, stored, and documented has 
changed over time. Schools and districts do not always enter data cleanly and 
we do not catch all their mistakes. Bottom line: An external partner must be an 
expert data manager, strong data cleaner, and have the patience to work with 
data sets that were collected for one purpose — basic reporting — but now are 
being used for another, namely, program evaluation. 

External partners must also be able to translate research into digestible 
formats for a non-technical policy audience. Overwhelming the superintendent 
or commissioner with information simply because the research partner is not 
willing or able to express findings in an easily understandable brief winds up 
harming — not helping — the SEA research cause. 


Family Educational Rights and Privacy Act: Implications 
for Research Partnerships 

The Family Educational Rights and Privacy Act (FERPA) guides both 
SEAs and external researchers in using educational data and is 
only growing in importance with concerns over “big data” and how 
student information is used and shared. An SEA can only re-disclose 
student data in certain circumstances; they include partners studying 
outcomes of educational importance and interest to the state. This 
helps explain why research partnerships and studies need to align 
with an SEA’s priority policy areas. It also underscores how important 
it is for an SEA to have thorough documentation on how data are 
being used. SEAs should refer to the National Forum on Education 
Statistics Guide to Supporting Data Access for Researchers. 
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Research partners must also understand the SEA’s bureaucratic constraints 
and be willing to work with the SEA to address emerging challenges. When this 
understanding and willingness is weak or absent, SEA staff become frustrated 
and the partnership is less likely to be productive. SEAs need research partners 
who can serve as partners not only in research, but also in learning. They must 
be willing to help SEAs develop their processes for this potentially fruitful work. 

Funding the Partnership 

Building productive partnerships takes resources for both SEA staff and the 
external researchers. States can seek out partners to compete for funding 
through the Institute of Education Sciences (IES), foundations, or other grant 
sources. A growing number of grants are available to research partnerships, 
such as the Partnerships and Collaborations Focused on Problems of Practice 
or Policy research program initiated by the Institute of Education Sciences. 
These grants focus on SEAs and local districts finding research partnerships 
on mutual topics of interest and are an encouraging development for research- 
driven SEAs. The grants are designed to support a range of partnerships and 
large-scale evaluation of state and local programs. 


Regional Educational Laboratories as Research Partners 

The Regional Educational Laboratories (REL) work in partnerships with 
state education departments. In Michigan, we have partnered with REL 
Midwest on several research projects including understanding: 

• Which measures predict whether a student is on track for college 

• Key issues related to early-childhood education quality 

• Which methods are most reliable for evaluating educators 

• Which strategies and practices differentiate schools that are 
beating the odds from demographically similar schools that are 
persistently low performing 

These projects leverage the federally funded REL program resources 
to inform the state’s ongoing policy and program work. 
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To date, IES grants include: 

• Evaluation of State and Local Education Programs and Policies, first 
awarded in 2009 and totaling 17, including the Michigan Consortium for 
Educational Research (see p.51). Evaluates major state or local policy 
initiatives using rigorous methods to estimate program impacts. 

• Researcher-Practitioner Partnerships in Education Research, first awarded 
in 2013 and totaling 20. Targets researcher-practitioner partnerships and 
frequently serves as a precursor to successful bids for the larger evaluation 
grants above. 

• Continuous Improvement Research in Education, first awarded in 2014 
and totaling six. Helps states and districts with short-cycle implementation 
science grants to regularly evaluate a program or intervention in shorter 
time frames to enable more rapid course corrections and continuous 
improvement. 

In summer 2016, IES will award more grants in each category and run a 
special competition for evaluation of federal ESEA flexibility waivers. 

Timing is a challenge in leveraging grant funds; the grant cycle is often too long 
to help a state answer a timely policy question. For instance, if an SEA and its 
research partner apply for a grant in August 2014, they will not hear if their bid 
was successful until July 2015, with a start date between July and September 
2015. That means a nearly year-long lag before work can begin. Meantime, 
potential partner schools and districts have moved on, policy has shifted, and 
the imperative for an answer to the policy question may have disappeared. 

SEAs and researchers are challenged to pick questions and topics that will 
remain relevant in a year, plus figure out what work to do and how to fund it 
while they wait for an answer on a grant proposal. 

Developing state block grants to fund SEA research questions would enable 
more flexibility in individual research projects and would greatly benefit SEA 
and local district research partnerships. The SEA would be the grant recipient 
and therefore be responsible for both meeting quality research and partnership 
standards and reporting on progress made with the grant funds. The SEA 
would have discretion to develop requests for proposals to use grant funds 
and to select research partners. Ideally, the funds would be used for a mix 
of short and descriptive “rapid response” studies to respond to immediate 
policy questions as well as for long-term ongoing partnerships on broad policy 
areas. This would also support states in developing ongoing partnerships that 
can be quickly leveraged to respond to new policy research needs without the 
conventional lag time involved. 
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Even without grants, partnerships are still possible. The key challenge is 
identifying researchers positioned to conduct the research without securing 
new funds. In Michigan, we have successfully partnered with senior university 
researchers who have built-in access to the needed resources (e.g., research 
assistants, software) and are relatively free from political interference to pursue 
relevant policy questions. 

Ensuring Visibility and Use of Findings 

Ensuring that findings and research products are well disseminated is a critical 
piece of building productive partnerships. Elsewhere in this volume, we discuss 
the importance of building internal capacity to produce, interpret, and act on 
useable research. But SEAs also need research partnership-generated findings 
to have visibility; it helps state agencies demonstrate that they are using 
research to make decisions and are invested in having solid information or the 
“right answer” to a policy question. Many state agencies suffer from a public 
relations problem of sorts — as the regulatory agency, they are often seen as 
overly compliance-driven and mired in bureaucracy. When SEAs contribute 
research, data, and information to the policy conversation, or can support their 
initiatives and policy decisions with relevant and timely research, it helps create 
a common conversation around difficult policy topics and, ultimately, can help 
the agency successfully carry out policies. Researchers, for their part, need 
their work to be visible because their professional worth is often judged by their 
success in publishing research and having their results referenced in the public 
policy domain. Partnerships between SEAs and external research can help 
researchers achieve public interest in their work. 

It can be challenging for states to ensure this visibility. Researchers generally 
do not write for a policy audience; SEAs generally lack a communications or 
public relations arm aimed at disseminating research findings. Suggestions for 
SEAs include: 

* Focus on developing defined deliverables and timelines for each partner. 

Michigan requires partners to produce four types of deliverables: a 
policymaker-focused document (1 to 2 pages, key takeaways); an executive 
summary; a full report; and a technical working document. Massachusetts 
requires a four-page summary for all research findings. 

• Highlight with research partners the importance of descriptive statistics 
and graphical representations. Many researchers produce these as an 
afterthought of sorts on their way to the “real question,” but this is 
valuable information for SEAs. Ask research partners to produce short 
descriptive reports every three to six months as they work on the larger 
question. 
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• Set regular times for researchers to present ongoing findings within the 

SEA. Options include department-wide leadership team meetings, executive 
team meetings, office-specific meetings, or brown bag lunches. Do this 
several times a year to help break the information into manageable pieces. 

• Ask research partners for six-month updates outlining their ongoing 
work and deliverables. This allows the SEA to take an active role in 
dissemination, through public releases of information and internal and 
external presentations. 

• Request that research partners submit to major research conferences and 
include an SEA staffer as a co-presenter in sessions. This highlights both 
the partnership and the work. Discuss the possibility of co-authorship with 
SEA staff, particularly on policy briefs or white papers geared to more of a 
policy than academic audience. 

• Preserve researchers’ independence and their ability to publish. In 

Michigan, we request a time to review all external researchers’ results 
for appropriate use of data and FERPA compliance, as well as to arrange 
our internal messaging if the findings are going to be highly visible or 
potentially contentious. But we protect the researcher’s academic freedom 
and do not interfere with the publication of results. 
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Case Study: Michigan Consortium for Educational Research 

In 2010, the Michigan Department of Education entered into a 6-year, $6 million 
partnership with the University of Michigan and Michigan State University to form 
the Michigan Consortium for Educational Research. The consortium focused 
on two key questions: what is the impact of the Michigan Merit Curriculum 
and what is the impact of the Michigan Promise Scholarship. The partnership 
also had another aim — to build an ongoing, collaborative research partnership 
with Michigan’s leading research institutions and use this to beef up the state’s 
infrastructure and capacity to do research with external partners. A few lessons 
learned include: 

• How to handle related studies and researchers. Since the partnership 
focused on two research questions, initial data approvals were related to those 
questions, as FERPA requires. But, over time, both universities added new 
graduate students, postdoctoral fellows, and other interested faculty. While 
interested in the original questions, these parties were even more interested 

in the state administrative data and the chance to do relevant policy research. 
This led to adding research questions and related hypotheses — a new concept 
for us. From the state’s perspective, a partner got data to do a certain study 
and when that study was done, the partnership was over. But related questions 
arise all the time as research unfolds. As the consortium studied the Michigan 
Merit Curriculum and its impact on achievement, new questions surfaced. 

What about teacher mobility? What about teacher supply and demand? What 
about schools that open and close over the study’s life? We had to strike a 
balance that afforded the consortium enough flexibility to grow while also 
maintaining strict documentation to ensure we followed the rules around 
researchers needing to study the educational question for which they are 
approved, not any question of interest. 

• How to provide appropriate longitudinal files. The consortium was approved 
for certain data sets and received those data. But where previous partners 
had gotten a data dump and then done their analyses, this time we created a 
standard process for researchers to request a regularly updated longitudinal 
data. 

• How to deal with special requests. In the consortium’s desire to address the 
research questions with the most rigorous data available, researchers often 
wanted data outside our ‘normal’ data set. This was initially a source of 
confusion or even worry on the part of the SEA: Why did the researchers want 
address data? How can we ensure compliance with FERPA’s requirement that 
we release only the most-needed data? To address this, we assigned an SEA 
“case manager” to each of our key research partners and, conversely, asked 
the external researchers to assign a point person on their side. This helps us 
solve problems and facilitate unusual data requests. 
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GETTING THE MOST OUT OF A 
RESEARCH PARTNERSHIP 

Building and sustaining mutually beneficial, ongoing research partnerships 
is not simple. It requires both the SEA and the researcher to build trust and 
invest in the relationship. The benefits that accrue from these partnerships 
include high-quality, relevant research using the state’s longitudinal data; 
regular reports and feedback; data “products” like researcher-ready data files; 
and being able to deploy the data to drive policy, versus leaving it to sit in data 
warehouses for reporting use only. To reap these benefits, however, states must 
overcome some challenges: facilitating work between multiple bureaucracies 
across different timelines and senses of urgency about the work, as well as 
communicating results to the public, particularly when negative attitudes 
prevail toward key policy initiatives. 

To maximize partnerships, SEAs should: 

• Leverage institutional and personal relationships within the state. 

Spend time developing a professional relationship with researchers. Find 
individuals who are committed to the state and the use of state data to 
drive policy; this will help you work through challenges in SEA-research 
institution collaboration. 

• Think about the strengths of different research universities in your state. 

Some might be best at research techniques that use advanced quantitative 
methods and longitudinal data; others better at studying certain 
interventions and how they work; others stronger in behavioral research. 
Research institutions are known for different things. Build the partnerships 
around strengths. Don’t be afraid to “cherry pick,” taking the best each has 
to offer. 

• Establish multi-university partnerships cautiously. These have great 
potential, but also can compound difficulties in navigating relationships. 

If you want to partner with multiple universities, make sure you have 
fully committed individuals from each university, ideally with some 
demonstrated track record of working together. 

• Get buy-in from SEA people at all levels, from the executive to program 
offices. Articulate a clear vision and need for this work, then show some 
‘quick wins’ or early value that these partnerships generate. Make sure 
someone is in place to translate between researchers and program staff — 
someone who knows the language of each and can help make sure they do 
not talk past each other. 
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* Make sure the questions tackled are answerable. It would be great to 
know conclusively what instructional practices are being used in our lowest- 
performing schools and how those relate to student engagement and 
motivation — but that is very tough to measure and requires much additional 
data collection. Many worthwhile questions and studies are much harder to 
accomplish than others. Prioritize so you can show the partnerships’ value. 

* Be honest with researchers about SEA internal dynamics and politics and 
the limits of what the agency can do. Researchers will be better partners if 
they understand the lay of the land from the outset. 

* Encourage research partners to develop policy briefs in a “question 
driven” format and address key questions in a non-technical way. Keep the 

writing simple, clear, and to the point. Tennessee’s exploration of course 
enrollment patterns for high school students provides a nice example of a 
research summary designed for a general audience. 

* Set clear expectations, guidelines, and rules, particularly around partners 
presenting results and giving the SEA sufficient notice. Make sure external 
researchers state that their findings reflect the researchers’ work and 

not necessarily the views of the state education department. Involve the 
SEA communications office early on to determine concurrent or related 
messaging and ensure agency staff know when a release is coming. Ensure 
researchers understand this is not about control or censorship, but about 
the SEA being able to have a policy-relevant response. 

* Have SEA staff attend key research conferences, such as those held by the 
Association of Education Finance and Policy and the American Educational 
Research Association. 

Identifying, recruiting, training, and supporting external research partners 
provides many excellent opportunities for SEAs, although agencies must 
invest time and internal resources to develop and maintain an infrastructure 
to support these partnerships. But the benefits of having high-quality, 
independent research on major policy areas of interest outweigh the costs. As 
states have developed comprehensive longitudinal data systems, developing 
the concurrent infrastructure to use those data is of utmost importance and 
supports the SEA's ability to make smart, data-based policy decisions. 
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The following is an edited transcript of the conversation the editors Betheny Gross 
(Research Director, Center on Reinventing Public Education) and Ashley Jochim 
(Research Analyst, Center on Reinventing Public Education) had with Kathleen 
Dempsey (Director, North Central Regional Comprehensive Center), Caitlin Howley 
(Director, Appalachian Regional Comprehensive Center), and Paul Kohler (Director, 
West Regional Comprehensive Center). To listen to the audio, visit BSCPCenter. 
org. Readers interested in learning more about how one state featured in this 
conversation built SEA capacity for research should look at the Nebraska Blueprint, 
featured at the end of this conversation. 

Betheny: Many people talk about the value of evidence-building in state 
education agencies, but it’s often not obvious where states should start or how 
research can inform their work supporting districts and schools. Paul, can you 
get us started by telling us a little bit about where you have seen research add 
value to state education agencies? 

Paul: Sure, I’d be glad to do that, and let me as a way to get started, talk about 
a regional collaborative with four states, and then talk about what’s happened 
with that. The states I’m going to talk about (Arizona, Colorado, Nevada, and 
Utah) were all prompted by national initiatives such as Race to the Top, or the 
school improvement grants, or the ESEA waivers. Each passed legislation on 
educator effectiveness that set specific requirements and timelines for the new 
standard statewide systems of teacher and principal evaluation. It included 
using student achievement as a significant indicator of effectiveness, and 
this was new for each of these states. Teacher evaluation has both technical 
components that require validation and implementation, and components that 
need to be monitored for [implementation] fidelity. 

In 2012, the West Comp Center formed a community practice and convened 
state department education leaders, which included policymakers and 
stakeholder groups from the four states to address these issues collectively 
and use available expertise and experiences to inform the design of these new 
evaluation systems. Once their frameworks were developed and adopted, it 
was apparent that the studies of these systems would need to be conducted. 
The West Comp Center partnered with a regional lab, in this case the Regional 
Education Laboratory West, to provide assistance to the Arizona Department of 
Education to study a pilot of the evaluation model in the selected districts. The 
findings from year one of the pilots informed changes to the program in year 
two with the West Comp Center and the REL West supporting the translation of 
findings into practice. 

After the pilot was completed, Arizona Department of Ed turned its attention 
to gathering information on how districts were using the data from teacher 
evaluations to inform the decisions related to professional development or 
assignment, compensation, remediation, retention, as well as identifying 
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teacher leaders. In Arizona, five districts participated in the study to 
understand how their districts were using the evaluation system and whether it 
aligned with the state policies and best practice guidelines. 

Betheny: Why do you think [the collaboration] was so effective in this case? 

Paul: [T]he three-year collaboration, and, by the way, it continues now into the 
fourth year, is an example of a strong partnership between the state education 
agency, a regional lab, which offered guidance in research design, methodology 
analysis of data, and the West Comprehensive Center, which provided technical 
assistance in the design and implementation of this system, and in using 
research findings to inform practice. The focus has been to help the state adopt 
a thoughtful, evidence-based improvement approach to implementation, and 
it has been shared work, collaboratively with defining questions of interest, 
as well as data and methods needed to address them. Then after data are 
collected by Arizona Department of Ed and the West Comprehensive Center, 
and analyzed by the regional lab, the preliminary findings were reviewed 
internally and the implications discussed. 

The collaboration has resulted in four of the lab studies and they are available 
on REL West’s website. Two have been published, one is in final editorial 
review, and another is now entering peer review. The Arizona Department of 
Ed is currently sharing the available results of these studies to districts across 
the state, not just the original five, as well as the state board of education, 
and they recently convened a task force by revisiting the state framework 
and timelines. In this case, the Arizona Department of Ed was successful at 
identifying changes that needed to be made in the model, capacity issues at 
the district level and at the state, by the way, early implementation strengths 
and the needs for improvement, and in making needed adjustments in a 
continuous improvement cycle. 

Arizona was able to take advantage of the complementary services and 
collaborations between the regional lab and the West CC and use the findings 
to inform their decisions, and we did this and we continue to support it 
because the state agency itself needed that assistance and really didn’t have 
the capacity to do that kind of work. 

Betheny: Caitlin, Paul’s discussion shows how fruitful good research can be for 
SEAs, but it also highlights how complex it is to get the pieces together. You’ve 
noted that many state agencies are hungry for research. In your experience, 
what do you think is holding SEAs back from building the requisite analytic 
capacities? 

Caitlin: Unfortunately, although lots of state education agencies really want 
good systematically collected information, they face some pretty substantial 
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obstacles to their ability to perform those analyses themselves. For example, 
the majority of their work tends to focus on implementing, often really 
ambitious, programs. Much of their knowledge and skill and energy ends up 
concentrated on crafting policies and guidelines that will work, standing up new 
programs, managing communication about new efforts and helping districts 
and schools with new implementations. 

At the same time, adding fuel to the proverbial fire, state agencies facing a 
proliferation of responsibilities including new federal requirements and work 
that emerges from the continually shifting policy environment that they are 
working in. Given the precedence of these responsibilities and pressures, 
there’s relatively less time and energy available for analytic work. 

Another challenge is that SEA staff are generally hired for their content 
knowledge, their management skills, their familiarity with federal education 
regulations, etc., not as often for their research capacities. While many states 
do have research offices, they often are overburdened with requests for help, so 
they may not be available to answer staff’s questions. 

A third challenge: lack of, or depleting, funding. For example, in the four states 
that Appalachian Regional Comprehension Center serves, education funding 
from the state remains lower than it was before the recession in 2009. Those 
kinds of funding losses make education more difficult all-around and money for 
research and evaluation is often not the highest priority. 

Betheny: Kathleen, I know that you have had the experience of working with 
Nebraska in overcoming some of these challenges. Can you say a little bit 
more about how states you’ve worked with are meeting the challenges Caitlin 
discussed? 

Kathleen: The North Central Comprehensive Regional Center worked 
with Nebraska Department of Education starting in 2012. The Nebraska 
Department of Education had received a three-year state longitudinal data 
system grant, and they had four goals with this grant. One of the goals was 
to provide a data analysis tool to districts that used multiple data sources 
to produce reports for local decision makers. Another goal was to provide a 
statewide system of data analysis, professional development for every district. 
The third goal was to build a research and evaluation, or an RE, team. Then, of 
course, they needed to be able to sustain this grant. 

North Central Comprehensive Center was asked to help NDE with this third 
goal, and that was to build a research and evaluation team. Because of some 
internal staffing transitions, NDE had lost the expertise necessary to establish 
this blueprint for this new data team, and so NCC was called to support 
them with that. What we did was to help bring in someone with some strong 


56 


THE SEA OF THE FUTURE | VOLUME 5: Building Agency Capacity for Evidence-Based Policymaking 



Evidence-Based Policymaking: A Conversation With the Regional Comprehensive Centers 


experience and background in working with research teams, and that was one 
of our REL directors. Through that experience, as far as having that expertise, 
he started work with that team to think about the course of action that they 
might take, and so some milestones were set up for the work. 

The first one was to determine how the data and research team at Nebraska 
would actually be positioned within their department. The second goal was to 
determine the mission, the vision, and the values of this team. The third goal 
was to determine the priorities for the team. The fourth one was to identify the 
roles and responsibilities of what was called a cross-team group. As they were 
putting this data research team together, they actually had people from across 
the department participate to help establish what the mission and vision of 
this team would be. Finally, to create a blueprint document that was to guide 
the work of the data research team. 

We were so glad to be able to work with them in this way, and within the year 
we were able to help them set up their team. They, in fact, did identify their 
mission, vision, and values. They set up priorities, meaning what they would 
do in-house, what they should outsource, when they might work with a partner, 
and all of those kinds of things just to kind of set up their working system, 
their processes and procedures. They got their roles and responsibilities set up 
and all of those things within that first year. They had a blueprint for how they 
would work as an organization. 

We continue to work with them. In fact, in this last year, we’ve been helping 
them put together a curriculum for how they will work with internal Nebraska 
Department of Education staff to help them be able to better use data to 
support their own decisions. 

Betheny: All three of you have really pointed to the fact that the lack of 
research capacity isn’t because states lack the desire to do the work. There’s a 
lot of interest out there in our SEAs, and they’re just sort of coming up against 
some challenges. Caitlin and Kathleen, what are other ways that states can 
tap the regional centers for more help when it comes to leveraging research to 
support their work? 

Caitlin: In our experience, state education agencies definitely want research. 
Not necessarily the “what works” experimental or consumer reports kind 
of research, but often more descriptive analyses of how certain groups of 
students are performing or how various programs that they’re running are 
implemented and with what results. What other state education agencies are 
doing to address shared issues or achieve similar goals. Given these kinds of 
needs, Regional Comprehensive Centers are really well positioned to provide 
technical assistance around analytical needs. 
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For example, the Appalachian Regional Comprehensive Center has assisted 
one state to plan, implement, and assess outcomes from a pilot of formative 
assessment instructional practices implemented in several school divisions, 
and is helping the state to use findings from that pilot to plan for scaling up. 

In another state, our team provided what’s essentially evaluation capacity- 
building services, to help the state plan, administer, and analyze results from 
a survey about which teacher performance assessment the state should begin 
using. 

Other examples include technical assistance to a state board of education 
to identify, review, adapt, administer, and analyze their first self-evaluation. 

In another state, we’ve helped the state education agency to investigate how 
teachers use that state’s online professional development system to better 
inform decisions that they were planning to make about whether or not or how 
to continue the use of that online professional development system. 

Other kinds of analytic assistance we’ve provided include help with the 
development of logic models so that states can monitor the progress of 
various initiatives, or assistance with creation of crosswalks to compare various 
state policies, so even as Regional Comprehensive Centers help state education 
agencies build their capacity to implement important new reforms, we also 
help enhance their capacity to access and use research to support their 
implementations of such reforms. 

Kathleen: Certainly, there are many ways to help our SEAs use and get the 
research that they need to make those decisions. The North Central RCC 
is helping the Wyoming Department of Education to find out about what’s 
happening in other states and in other locales as far as schools that serve 
populations with large numbers of Native Americans. We’re working with the 
department to identify schools that serve a high population of Native American 
students, and that are achieving at high rates regarding student attendance, 
graduation rates, and academic achievement in English language arts and 
mathematics. 

To do this, we turn to our REL Central partners, and they’ve been a part of the 
discussions with Wyoming Department of Education, so we worked with them 
to identify schools and to help us craft survey questions. There have been more 
than 100 schools that have been identified across 10 states, and we’re getting 
ready to send out surveys, and hopefully we’ll get a good response on that. 
Wyoming is looking forward to the information, so they’ll inform next steps 
and certainly it’s aimed at thinking about better ways to serve Native American 
schools in Wyoming. 

Paul: I think the point that needs to be made is that state agencies need a 
research component as they work to implement some of these policies. The 
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districts are asking for it. State policymakers, both at the legislature and the 
state board, are asking for it. 

You’ve already heard in this conversation that while the states are interested 
in doing it, they don’t always have the capacity to provide that service. The 
West Comp Center Department of Education does a lot of turnaround school 
work by training leaders and their teams, and we are following up those state 
initiatives with case studies that are being written by our staff at the Comp 
Center, but also with the help of the REL, to find out what the impact of the 
turnaround leadership work is in school districts that are sponsored by the 
states. That’s another example where the credibility of a research group such 
as an REL can be very helpful to a state, again providing a service that they 
may not be able to do, but providing a third party credibility and validations to 
some of the work. 


THE NEBRASKA BLUEPRINT 

Context: The Nebraska Department of Education (NDE) received a three-year 
State Longitudinal Data System (SLDS) grant in July 2012. The grant goals 
were to provide a data analysis tool to districts that uses multiple data sources 
to produce reports for local decision makers, provide a statewide system of 
data analysis professional development for every district, build a research and 
evaluation (R&E) team in NDE, and sustain the SLDS. NDE requested NCCC 
assistance with the third grant goal. Due to internal NDE staffing transitions, 
NDE lost the expertise necessary to establish a blueprint for the new team. 
NCCC support intended to help NDE create a Data, Research, and Evaluation 
(DRE) Team to support SLDS work in collaboration with the NE research 
community (i.e., higher education institutions, Education Service Units [ESUs], 
and schools), assist NDE with R&E needs, coordinate the research community’s 
involvement in the NDE Data Analysis Cadre (a combined NDE and ESU 
cadre team), and disseminate research findings to NDE and the NE research 
community. NCCC assistance on this project included supporting a McREL 
Senior Fellow to collaborate with the DRE Team director on how to develop a 
blueprint to guide the formation of the DRE team. Progress toward milestones 
includes: 

1. Determine how the DRE team will be positioned within NDE. 

During a February 2013 meeting, participants decided the DRE Team 
should be reflected in the NDE organizational chart to highlight their 
role as an NDE supporting unit. DRE Team work is expected to help NDE 
become more intentional about integrating work across NDE. NCCC 
assistance included supporting the attendance of the McREL Senior Fellow 
at two meetings and providing meeting facilitation. 
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2. Determine the mission, vision, and values of the DRE team. 

NCCC supported participants in identifying a mission, vision, and values 
by facilitating the meeting in a way that all voices were heard. The revised 
mission emphasizes new R&E functions and expands the scope of NDE 
work. The DRE Team will collect and report on data necessary to comply 
with state and federal legislation and data related to other NE research 
community priorities. 

3. Determine the DRE team work priorities. 

A third priority for the meeting was to establish parameters for the DRE 
team to ensure quality work. Questions guided the discussion of work 
parameters: what work do we do in-house, what work should we outsource, 
and what work do we partner to accomplish. NCCC supported these 
discussions by suggesting productive meeting structures for discussions, 
taking notes, and facilitating discussions. 

4. Identify the role and responsibilities of the cross-team group. 

The cross-team group was established to provide guidance to the DRE 
team, including the new director. Participants will meet to support the DRE 
team. NCCC supported this milestone by providing productive meeting 
structures. 

5. Create a blueprint document to guide the work of the DRE team. 

After each meeting, NCCC compiled a blueprint document with detailed 
notes on meeting discussions by topic. This document is intended to 
provide guidance to the DRE team. 

Another milestone was to determine the agenda and structure for two cross- 
team meetings. NCCC, McREL Senior Fellow, and DRE team director held 
several meetings to plan the cross-team meetings goals and the meeting 
structure to get input from all participants. The planning insured that the 
meetings were productive and that the milestones were met. 

Outcomes: Progress toward the project outcomes was made in Year 1. The 
DRE team role was formalized, including policies and procedures to guide their 
work. 

NCCC continues to assist NDE to build internal SEA capacity for using data to 
inform decision-making. 
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About the SEA of the Future Series 

The SEA of the Future is produced by the Center on Reinventing Public Education (CRPE), 
a non-partisan research and policy center at the University of Washington developing 
system-wide solutions for K-12 public education. Through research, policy analysis, and 
technical assistance, CRPE focuses on how states can better support school and district 
improvement. 


About the BSCP Center Partners 

The SEA of the Future is a product of the Building State Capacity and Productivity 

Center (BSCP Center), which focuses on helping state education agencies (SEAs) 
throughout the country, as they adapt to increased demands for greater productivity. As 
state departments of education are facing the daunting challenge of improving student 
performance, the BSCP Center provides technical assistance to SEAs that builds their 
capacity to support local education agencies (LEAs or districts) and schools, and to 
the other 21 regional comprehensive and national content centers that serve them, by 
providing high-quality information, tools, and implementation support. The partners in 
the BSCP Center are Edvance Research, Inc., the Academic Development Institute, and 
the Edunomics Lab (Georgetown University). 

Edvance Research, Inc. www.edvanceresearch.com | Founded in 2005, Edvance 
Research, Inc. (Edvance) is a mission-driven, women and minority owned, small 
business, nationally recognized for innovative and trusted expertise in education. A 
proven leader, specializing in collaborative research and development, evaluation, 
project management, assessment, research, large-scale initiatives, marketing and 
market research, training, and building capacity to use research, Edvance is committed 
to providing exceptional value to clients through outstanding quality and best practices. 
Edvance has held contracts with state education agencies, the Texas Education Agency, 
private institutions, foundations, and, most recently, a Regional Educational Laboratory 
for the U.S. Department of Education. Edvance is headquartered in San Antonio with 
offices in Austin, Texas. 

Academic Development Institute (ADI) www.adi.org | The Academic Development 
Institute (ADI) focuses on helping state education agencies (SEAs) throughout the country, 
as they adapt to increased demands for greater productivity. As state departments of 
education are facing the daunting challenge of improving student performance, the 
BSCP Center provides technical assistance to SEAs that builds their capacity to support 
local education agencies (LEAs or districts) and schools, and to the other 21 regional 
comprehensive and national content centers that serve them, by providing high-quality 
information, tools, and implementation support. 

Edunomics Lab at Georgetown University www.edunomicslab.org | Edunomics 
Lab is a university-based research center focused on exploring and modeling complex 
education finance decisions. The newly formed center is in the McCourt School of 
Public Policy at Georgetown University and focuses on public spending for K-12 and 
higher education. The center tracks public funds through the system to the point of 
service to examine the different policy decisions and their effects on the allocation of 
resources across students and services. 
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